From: Dmitry Rogozhkin <dmitry.v.rogozhkin-at-intel.com@ffmpeg.org>
Date: Tue, 21 Nov 2023 05:57:32 +0000 (-0800)
Subject: avcodec/decode: validate hw_frames_ctx when AVHWAccel.free_frame_priv is used
X-Git-Tag: archive/raspbian/7%8.1-3+rpi1~1^2^2^2^2^2^2^2^2^2~1
X-Git-Url: https://dgit.raspbian.org/%22http:/www.example.com/cgi/success//%22mailto:kde%40ewsoftware.de/%22/%22http:/www.example.com/cgi/success/%22mailto:kde%40ewsoftware.de/%22?a=commitdiff_plain;h=709935251dc9d01ba8bed8024acdae127161850a;p=ffmpeg.git

avcodec/decode: validate hw_frames_ctx when AVHWAccel.free_frame_priv is used

Validate that a hw_frames_ctx is available before using it for
the AVHWAccel.free_frame_priv callback, and don't require it to
be present when the callback is not in use by the HWAccel.

v2: check for free_frame_priv (Hendrik)
v3: return EINVAL (Christoph Reiter)
v4: better commit message (Hendrik)
v5: fix typo with missed frames_ctx (Lynne)

See[1]: https://github.com/msys2/MINGW-packages/pull/19050
Fixes: be07145109 ("avcodec: add AVHWAccel.free_frame_priv callback")
CC: Lynne <dev@lynne.ee>
CC: Christoph Reiter <reiter.christoph@gmail.com>
Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>

Gbp-Pq: Name 0002-avcodec-decode-validate-hw_frames_ctx-when-AVHWAccel.patch
---

diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index ad390213..2cfb3fcf 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -1838,17 +1838,26 @@ int ff_copy_palette(void *dst, const AVPacket *src, void *logctx)
 int ff_hwaccel_frame_priv_alloc(AVCodecContext *avctx, void **hwaccel_picture_private)
 {
     const FFHWAccel *hwaccel = ffhwaccel(avctx->hwaccel);
-    AVHWFramesContext *frames_ctx;
 
     if (!hwaccel || !hwaccel->frame_priv_data_size)
         return 0;
 
     av_assert0(!*hwaccel_picture_private);
 
-    frames_ctx = (AVHWFramesContext *)avctx->hw_frames_ctx->data;
-    *hwaccel_picture_private = ff_refstruct_alloc_ext(hwaccel->frame_priv_data_size, 0,
-                                                      frames_ctx->device_ctx,
-                                                      hwaccel->free_frame_priv);
+    if (hwaccel->free_frame_priv) {
+        AVHWFramesContext *frames_ctx;
+
+        if (!avctx->hw_frames_ctx)
+            return AVERROR(EINVAL);
+
+        frames_ctx = (AVHWFramesContext *) avctx->hw_frames_ctx->data;
+        *hwaccel_picture_private = ff_refstruct_alloc_ext(hwaccel->frame_priv_data_size, 0,
+                                                          frames_ctx->device_ctx,
+                                                          hwaccel->free_frame_priv);
+    } else {
+        *hwaccel_picture_private = ff_refstruct_allocz(hwaccel->frame_priv_data_size);
+    }
+
     if (!*hwaccel_picture_private)
         return AVERROR(ENOMEM);